Efficient algorithm for regularized risk minimization

The recently proposed Optimized Cutting Plane Algorithm (OCA) is an efficient method for solving large-scale quadratically regularized risk minimization problems. Existing open-source library LIBOCAS implements the OCA algorithm for two important instances of such problems, namely, the Support Vector Machines algorithms for training linear two-class classifier (SVM) and for training linear multi-class classifiers (MSVM). In this thesis we implemented an extended version of the LIBOCAS library which is able to solve the risk minimization problems with a more generic risk function. In particular, our solver allows the risk to be a generic piece-wise linear function. We give necessary mathematical background of the OCA algorithm and we describe details of our implementation. We show how to use our generic library to implement solvers for the SVM and the MSVM algorithms. We experimentally compare our implementation with the LIBOCAS on several benchmark data sets. The comparison shows that our library obtains exactly the same solution as the LIBOCAS requiring a comparable convergence time while being able to deal with far more generic risk functions.